Directory Synchronization using Concurrency in Go
19 Mar 2017
Tags: go golang concurrency

Zhibin Cai
Technical Overview
zcai108@syr.edu
zhibincai.com


* This presentation is about...
- Go language
- The *concurrency* feature in Go
- Final Project: Directory Synchronization just like Dropbox
# Put a dropbox logo here 
# to illustrate about the Synchronization



* Hello World
.play code/helloworld/main.go



* Go Basics
.code code/basic/main.go /start OMIT/,/end OMIT/


* Methods
.play code/methods/main.go /start OMIT/,/end OMIT/

: The recever is explicit, there is no `this` in go lang



* Let's talk about concurrency



* It's all about concurrency
- What is concurrency?
- Is concurrency also called parallelism?
- What is the feature of concurrency in Go?
- What is it good for?
- How am I going to use it in the project?





* Example of Go concurrent code



* Infinite function
.play code/blah/main.go /start OMIT/,/end OMIT/




* Goroutine
The `go` statement runs the function on the fly in the same address space and does not block the called.
This is a go routine.
It just like you add `&` at the end of a shell command.
You just don't care.
.play code/ignore_blah/main.go /start OMIT/,/end OMIT/




* Channels
Channel is an approach of communication in go for goroutines to exchange data back and forth.
.code code/channel_syntax/main.go




* Using channel to get value
.play -numbers code/channel_blah/main.go /start OMIT/,/end OMIT/






* Another Example: ping-pong
.play code/pingpong/main.go /start OMIT/,/end OMIT/








* Synchronization
When the main function executes `<-c`, it is blocked until a value is sent from that channel.
Also in the `blah` function executing `c <- value`, it is also blocked until someone gets the value from itself.

Both of them are blocking operation.







* The Go Approach
Don't communicate by sharing memory, share memory by communicating.





* Go Concurrency Patterns




* Pattern: Generator (Function that returns a channel)
.play code/generator/main.go /start OMIT/,/end OMIT/





* Generator works like a service
One function that returns a channel can be also regarded as a software service that run some job in background.
.play code/service/main.go /start OMIT/,/end OMIT/

- In this example, we have two independent instances of channel(service).





* Fan-in function
.play code/fanin/main.go /start OMIT/,/end OMIT/
.image img/fanin.jpg _ 400





* Select
- A *Select* is like a `switch` statement in which the cases are communications.
- A simple example uses a second channel to tear down the server.

    select {
        case v1 := <-c1:
            fmt.Printf("received %v from c1\n", v1)
        case v2 := <-c2:
            fmt.Printf("received %v from c2\n", v1)
        case c3 <- 23:
            fmt.Printf("sent %v to c3\n", 23)
        default:
            fmt.Printf("no one was ready to communicate\n")
    }








* Fan-in Function example
.play code/faninAgain/main.go /start OMIT/,/end OMIT/
.image img/fanin.jpg _ 400




* Fan-in Function using Select
.play code/faninSelect/main.go /start OMIT/,/end OMIT/
.image img/fanin.jpg _ 400






* Quit for a server
We can use select and tell the server to stop when there is data sent to the quit channel.
.code code/select/main.go /start OMIT/,/end OMIT/




* Channel works just like Chinese Whisper Game

.image img/chinese.jpg
.caption Reference: _Chinese_Whisper_Game_ by Rob Pike



* Daisy chain
We can run thousands of goroutines at once...
.play code/chinese/main.go /start OMIT/,/end OMIT/
.caption Reference: _Daisy_Chain_ by Rob Pike





* How to use Go concurrency in Directory Synchronization



* Design the architecture first!
- Let's *fake* the services at the beginning.
- We called it `fakeDropbox`.


* Fake Dropbox Example

What is it? 

Given a series of files, sync the content with the corresponding files on the cloud.

We only cares about the delta of the files.
- If we are editting the plain text, we send the changes of the files.
- If we are handling the binary files, simply deal with the whole binary file.


Two major actions:

1. `SendChanges("Images")`
2. `GetChanges("Images")`


* The process of synchronization
.image img/dropbox.png




* Dropbox sendChanges
We can mock this `SendChanges(directory)` function to *simulate* the sync function.
.code code/dropbox1/main.go /start OMIT/,/end OMIT/




* Fake Dropbox service v0.1 beta
This dropbox service only trigger the sync *sequentially*.
.code code/dropbox1/main.go /start2 OMIT/,/end2 OMIT/
.play code/dropbox1/main.go /startMain OMIT/,/endMain OMIT/



* Fake Dropbox service v1.0
.code code/dropbox1/main.go /start2 OMIT/,/end2 OMIT/
New version using a fan-in function
.play -edit code/dropbox2/main.go /start2 OMIT/,/end2 OMIT/


* Using Go with concurrency
- No mutexes / locks. No callbacks
- Cheap goroutines
- Communicate between goroutines using channel like a human


* Future module/architecture
.image img/architecture.png
